Ty: Lossless Data Compression for Analytics-driven Query Processing

ARKATKAR, ISHA. ALACRI2TY: Lossless Data Compression for Analytics-driven Query Processing. (Under the direction of Nagiza F. Samatova.) Analysis of scientific simulations is highly data-intensive and is becoming an increasingly important challenge. Peta-scale data sets require us to look for alternative ways of performing query-driven analyses. This thesis is an attempt in the direction of query processing over losslessly compressed scientific data. We propose ALACRI2TY (Analytics-driven Lossless dAta Compression for Rapid In-situ Indexing, sToring, and querYing), which at its core consists of two components: lossless compressor and query processing engine over compressed data. ALACRI2TY’s compression component performs compression of double precision scientific data by unique value-based binning. Based on significant bit splitting, ALACRI2TY improves compression ratios over general-purpose compression utilities. It then indexes the metadata about the compression rather than the data to enable light-weight index storage. The query processing engine answers range queries over this compressed data with a low degree of unnecessary decompression. ALACRI2TY’s methodology involving compression and binning enables (1) Indexing with a total storage requirement (data+index) of less than 135% (versus 200-300% in existing scientific database systems); (2) Data access at multiple precision levels of detail necessitated by the varying sensitivity of analytical kernels (e.g., low-precision for histograms and descriptive statistics, medium-precision for clustering, and full-precision for Fourier analysis); (3) Robust performance across univariate as well as multi-variate query constraints via efficient bitmapbased aggregation of partial results. Altogether, these capabilities yield a multi-fold improvement in query response time over state-of-the-art systems such as FastBit, MonetDB, and SciDB when tested on several realworld data sets from scientific simulations and using the high-end compute clusters and Lustre file system at Oak Ridge National Laboratory. c © Copyright 2012 by Isha Arkatkar